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NOTE 



This volume is a supplement to the main report 
of this study. Volume I: Description of Studj^ 

Methodology and Findings . Although data reported 
herein are of singular value, we do not believe that 
they can be interpreted meaningfully without the 
reader’s full understanding of the survey method- 
ology (its lo^e and limitations) , which is 
reported in considerable detail in Volume I. 
Therefore, we recommend that this volume be 
used only in conjunction with Volume I. 
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I. INTRODUCTION 
(from Volume I) 



The purpose of this study was to evaluate the quality and utility of NCKC 
information analysis products, including ERIC clearinghouse products, PRHI 
reports, and EMC bib l-f ographies . * This project was supported by the Oltieo 
of Program Planning and Evaluation in the U.S. 01 t ice ol Education (USOE) , 
and Xv'as conducted over a 12-month period from July through June 1972. 



As stipulated by USOE, the study was -.to focus only on the products, and not 
on the management process by v?hlch they were conceived and prepared. Withi 
this limitation of scope, the planning and conduct of the study were guided 
by two major goals: 

• To develop data from a cross-section of educators 
regarding their level of familiarity with, and 
judgments on the quality and utility of, NCEC 
information analysis products. Although based on 
specific documents, the data would be analyzed in 
relation to characteristics of both the user 
population and the documents so the results could 
assist USOE in developing policy-related guide- 
lines for their future information analysis 
activities. 

• To assess the SDC survey methodology so that a 
well founded plan for continuing evaluations of 
NCEC products could be recommended. 

An outline of specific issues addressed in the study follows a brief dis- 
cussion of the products and their originating units. 



These acronyms are used throughout the report: 

NCEC: National Center for Educational Communication 

ERIC: Educationax Resources Information Center 

PREP: Putting Research into Educational Practice 

EMC: Educational Materials Center 



A. 



NCI'C INFORM/iTION AILALYSIS TRO DUCTS 



Thi’ lUrrattiri' of and practice is synihesizeii in three major types ui 

XCLC i’.'.f erm;it i ini .'malys i s products: ERIC clearinghouse products, EMC 

bib 1 iograi lies , and PREP reports. A brief background on each of these product 
groups is provided in the fol loving sections. 



1. i'.RlC CLEiXKlXGiiOUSE PRODUCTS 

Each of the 19 ERIC clearinghouses acquires, screens, indexes, and abstracts 
the published and unpublished literature in its respective content area. The 
products of these efforts are published in ^e^a^.in. Educati^ and Curr e^ ^ 
Tiidex to Jo urnals in Education . In addition, the clearinghouses nroduce a 
number of special information analysis products on subjects relevant to then 
scope of coverage. These products represent not only a range of product types 
(e.g., biblicgraphies and reviews), but a number of forinats (e.g., newsletters 
and monographs) and dissemination media (e.g., journal articles and chapters 
in books). Adequate definition of "ERIC information analysis product," 
therefore, was a challenging part of the initial project work. Through a 
process of analysis and refinement, a decision was made to include three major 
types of products: bibliographies (citations only, citations with abstracts, 

and citations with annotations), reviewt a.nd state-of-the-art papers, and. 
practical guidance papers. 

The various special information analysis products have been prepared by tht. 

ERIC f'learingnouses for the past 6 years. The steady growth of this program 
is illustrated in the yearly increments of publications cited in ERIC Product _ s , 
an annual bibliographic, publication of the ERIC Clearinghouse on Library and 

Information Sciences. These figures are; 

1967- 1968 149 citations 

1968- 1969 240 citations 

1969- 1970 366 citations 

1970- 1971 9l6 citations 

Although some products are channeled through the professional journal literature, 
most of the products under study were originally available as monographs, for 



which initial press runs were made for clearinghouse distribution, followed by 
distribution through the ERIC Document Reproduction Service (LDRS) . 

Fach year, the clearinghouses submit budget plans to NCtC that outline the 
number and types of information analysis products proposed fur the ensuing year. 
Approximately 40 to 45 percent of the total budget tor each clearinghouse is 
for the information analysis program. The total NCEC budget for this program 
is approximately one million dollars. 

2. PREP IfEPORTS 

The Targeted Communications program of the NCEC's Division of Educational 
Extension Systems (formerly the Division of Practice Improvement) is the 
foundation for interpretive summaries called PREP reports. Projects are funded 
through contracts and grants to interpret research and development findings 
that have a potential for improving educational practice, particularly in USOE 
priority areas. The program specifies that the projects should be described 
in non- technical language and in such a way that they will meet the needs of 
specific, non-research audiences. 

The actual PREP reports are created within NCEC as a byproduct of the Targeted 
Coraitunicatiors program. (The PREP report budget is a small fraction of the total 
budget for the Targeted Communication program.) Although these reports use 
much of the material— verbatim— from the final project reports, some formal 
or organizational changes are sometimes made. For example, a given project 

report may be repackaged into more than one PREP report. 

% 

To date, 30 PREP reports have been prepared. Approximately 300 copies of each 
are sent by USOE to state education agencies, who in turn have primary 
responsibility for distribution to appropriate target audiences. These reports 
are also made available through the U.S. Government Printing Office and the 
ERIC Document Reproduction Service. 

Workshops and conferences for disseminating information are often a part of 
Targeted Communication projects. Originally, several workshops were to have 
been evaluated in this study. However, investigation revealed that evaluations 



had already been conducted on most ot these wurkshops, by the sponsors, 
immediately lollowlnK the workslioj^s . Moreover, since cousideraole i ina- ii.ic 
elapsed since the workshops had occurred, there WaS serious doubt Ltiat tne 
participants would remember the vcorkshop content or be able to ideniily ils 
impact . 



3. i:mc iHuuoihiAPim'.s 



Bibliographic reporting services provided bv the Educational Materials C 
(now a part of the expanded Educ-ational Reference Center) draw upon a 
collection of appi - ximately lb, 000 textbooks, children's hooks, and pro: 
t'ducaiion materials provided by publishers on a "permanent loan" basis, 
of th.is collection is housed at the Federal City College in Wash i ngt 'm , 



ell t e r 



t' b r» i c3 11 1 : 

Mo.st 
D . C . 



f 



Over a period of 11 years, from the time it was the Educo.tional Materials 
Laboratory, the Center has prepared 54 b ib 1 iograph i es . Although changes in 
formats have occurred over the years, the purpose has always been the same; 
to provide educators with bibliographic information on the particular materials 
in the Center's collection. 



EMC bibliographies are distributed free by USOE .nid sold ti^rough tlie U.S. 
Government Printing Office. 
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iii. Request for Proposals, USOh identified a number of 
addressed in the study. Per each product in the sample 
tempt to answer questions such as the following. 



issues that needed 
, the study was to 



to 



• Is the intended audience aware of the product’s existence? How 

manv have read it? 

% 

• How’ Xv’as it received by its intended audience? 



. Wiat is thu quality of the product as perceived by intended users 
and others qualified to judge? What were its strengths and 

de f iciencies? 



• Did it accurately summarize and defensibly intv^.rpret the 
relevant literature? 



• Do the bibliographies direct the user to the appropriate 
literature? 

• How useful was the product to intended users? What use did they 
make of it (e.g., w’as it considered general information; was 

it used in solving a specific educational problem)? 



• Did it satisfy user needs? What needs? 

• Did the product reflect old, current, or original approaches 
to problems? Are the problems timely? 

• What impacts can users report on practice as a result (even 
a partial one) of reading the publication? 

• How do qualified experts rate the report as a contribution to 
information distillation of the educational literature? 

• I^mat were the "side-effects" of the document (e.g., use by 
persons other than the intended audience, utility other than 

intended, etc.)? 

Each ot these, and other related questions, were carefully considered in the 
design of the survey plan, particularly in the selection of survey participants 
and the development of survey instruments. Features of the SDC approach are 
summarized in the next section. 
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c. 



STUDY AI F 'AUi 



The overall plan for the study was to obtain an appropriately large number of 
qualit y— u till ty judgments on NCEC products from a sample representative of the 
educational community. Features of the study plan are highlighted below; 

• The Product Sample . A fairly large sample (146) of NCEC product 
was carefully selected to represent the major kinds of produv:ts 
and the major content areas. 

• The Two Surveys . Two surveys were conducted: 

- A General Field Survey, representing a broad cross- 

•ection of educators, and 

< 

- A Specialists' Survey involving individuals identified by 
their colleagues as being particularly well qualified to 
evaluate documents in the product sample. 

Participants for the General Field Survey were drawn in two 
ways : 



A rigorous sample — the random sample — was drawn from 
personnel listings of state education agencies, local 
school districts, junior colleges, colleges/universities, 
and USOE— supported research facilities. 

- A second sample — the non-random sample — was drawn from 
several available listings, including ERIC Clearinghouse 
mailing lists, state and local educational information 
center user lists, and ERIC Document Reproduction Service 
on-demand sales records. 

• The Survey Instrumei.ts . Four questionnaires were developed for 
the study: 

A Screening Questionnaire (Ql) was mailed to participants 
in the General Field Survey to identify educators familiar 
with NCEC products. . It included a special color insert 
of miniature photos of sample products. 



- A User Evaluation Questionnaire (Q2) was mailed to a 
selected group of respondents to the Screening Questionnaire 
who had read or skimmed products from at least one NCEC 
unit. Participants were asked to evaluate 10 documents, 
each of which was individually assigned on the basis of 
Screening Questionnaire data. A document representation 
(i.e., title page, table oi contents, and abstract or 
extract) was attached to each questionnaire. 

- A Non-user Evaluation Questionnaire (Q3) was mailed to a 
selected group of respondents to the Screening Questionnaire 
who reported having not read or skimmed NCEC products. 

An abbreviated form of the User Evaluation Questionnaire 
was developed to explore non-users^ potential interest in tiu- 
documents. Procedures and packages comparable to those 
of the User Evaluation Questionnaire were uied . 

- A Specialists' Questionnaire (Q4) was mailed to selected 
specialists. Some questions were comparable to those of 
the User Evaluation Questionnaire, but explored the 
quality dimension in more depth. Documents were 
individually assigned and each specialist evaluated an 
average of two or three documents. Complete copies of 
documents were provided. 

9 Data Analysis . Data from the four questionnaires were analyzed 
to relate to each of the study issues and questions. 

Several kinds of survey findings are reported: 

- Evaluation data from Readers, Non-Readers, and Specialists 
are displayed in individual document evaluation profiles. 

- Evaluation data from Readers are aggregated on documents 
for each product type, subject area, and user group, 

as well as by level of product exposure and level of 
effort involved in the production of the product. 

- Non-reader and Non-user data are reported in the 
aggregate for documents in each product group. 

- Specialists data are reported for individual documents 
and, in the aggregate, for each of the three product 
types . 



These and other special analyses are reported in one of tx<;o volumes of this 
report, as discussed in the next section on the organization of this report. 
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D. 



ORGANIZATION OF THE REPORT 



This study is reported in two volumes. Volume I describes the study objectives, 
reviex(7S the survey methodology, and reports general findings and conclusions. 

As a supplement, Volume II contains the basic evaluation data, from Specialists, 
Readers, and Non-readers, on each of the 146 documents in the product sample. 

V 

The next chapter in Volume I is an executive summary that is written to 
provide an overview of the entire study in capsule form. Chapter III presents 
a detailed account of the survey methodology, from the development of the 
product sample through the conduct of the General Field and Specialists’ 

Surveys. In Chapter IV, the respondent populations of these surveys are 
described by their various user characteristics. 

Chapter V begins the reporting on survey findings with a presentation and 
discussion of data regarding the respondent populations’ levels — both 
general and product-specific — of familiarity with NCEC information analysis 
products. A comparison of the two samples of the General Field Survey is made 
in Chapter VI, paving the way for the report in Chapter VII on the several 
aggregated data anax^’^ses. These analyses, and the conclusions and recommendations 
in Chapter VIII, arc organized by study objectives and issues introduced 
in this Chapter. 

Supplementary materials and tables are contained in several Appendices at the 
end of Volume I. The organization of Volume II is described next. 



II. INTRODUCTION TO THE "INDIVIDUAL DOCUMENT 
EVALUATION PROFILE" 



Individual document evaluation profiles are reported in a three-page display 
for each of the 146 documents in the sample of NCEC information analysis pro- 
ducts. These pages are: 

1. Title Page and Descriptive Data Report 

A 

2 . Reader and Non-Reader Evaluation Data 

A. for Practical Guidance Papers and Reviews 

B. for Bibliographies 

3. Specialists’ Evaluation Data 

A' sample set of these standardized reporting formats is provided in Exhibits 
1 through 3, with accompanying 'explanatory notes keyed to each element in the 

exhibits. 



Reader data are provided only on 58 products. For 50 documents, the sample 
size was 15 or more; in eight selected cases (i.e., selected to provide 
coverage of individual clearinghouses), the sample size was between 10 and 15. 













ERIC 






Explanatory Note for Exliibit li 
SAMPLE EVALUATION PROFILE COVER PAGE 

A. DOCUMENT DESCRIPTION 



1. Document No . SDC-assigned number. 

2. Citation . Includes title, author, series (if any), date, and ED or GPO 
number . 

3. NCEC Unit. Name of ERIC Clearinghouse. 

NCEC Product . PREP Report or EMC Bibliography. 

4. Product Type . Review, Practical Guidance Paper, or Bibliography. 

5. Subject Cluster. One of four general areas: Instructional Content , ^Edu~ 

cational AdmlnTstration and Services; Special and Other Educational Groups, 
or Higher Education. 

6. Level of Effort Index. High, Medium, or Low. (An asterisk indicates that 
no data were available and the median number of hours was assigned.) 

7. Le vel of Visibility Index . High, Medium or Low. (An asterisk indicates 
that no data were available and the median number of copies was assigned.) 

B. GENERAL FIELD SURVEY (User Evaluation Questionnaire) 

1. N. Number of respondents who reviewed the document. 

2. Familiarity . Percentages of respondent population for each level of fam- 
iliarity. (**Previously read/skimmed” group are hereafter called Readers, 
others Non- Readers . ) 

3. Recency of Reading . Percentages, within the Reader population, for each 
given time period. 

4. Comments. Drawn from both Readers and Non-Readers. Reader comments usually 
made relative to the "usefulness” and "impact" questions; others are general 
observations. Non-Readet comments are generally "other" explanations why they 
did not read/skim a product they had heard about or seen. A Respondent s role/ 
function, an abbreviation of one of the 15 general user groups, is underlined 
and precedes his comment. 

X C. SPECIALISTS* SURVEY 

i) 

1. N. Number of Specialists, either 2 or 3. 

2. Recency of Reading. A background item. (Specialists were not expected to 
have read documents prior to receiving complete copies of documents to re- 
view.) 

3. Comments. Drawn and synthesized from open-ended responses to quality- 

related plus any general comments. Each bullet represents a differ- 

ent Specialist. 
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EXHIBIT 2A: 



SAMPLE EVALUATION PROFILE PAGE FOR READER AND NON-READER DATA FOR 
REVIEWS AND PRACTICAL GUIDANCE PAPERS 

(Document (No.) continued) 






EADER EVALUATIONS (N- 



QUALITY 



O ! 
O 
© 



Coverage 
Up-to-dateness 
Organlzati<^ 
Writing 
Format 
Discussion 



Mean 



©= 



O 
© 



G 



Percentage 



Reference 

Mean 

( ) 

( ) 

( ) 

( ) 

( ) 

( ) 

Reference 

Percentage 



{ ft \elevance 

^J-^eed 



ength: 



UTILITY 



Mean 



Reference 

Mean 



f'ioV omparative usefulness 



Purpose of use 



= 0 



About right 


_% 


(. 




Too long 




(, 




Too short 


_% 


(. 





Obtain overview 

Look up facts 

Identify individuals 

Identify relevant 
literature 

Update knowledge 

Obtain new 
knowledge 



( 


) 


( 


) 


( 


) 






( 


) 


( 


) 


( 


) 


( 


) 


( 


) 



© 



IMPACT 



Percentage Reference Percentage 



Used to make decision 


% 


(_ 




Applied in my work 


% 


(_%) 


Used to give advice 


% 


(„ 




Examined other documents 


% 


(_ 




Consulted with author(s) or others 





(_%) 


Passed document on to colleague(s) 





(_ 





^ON-READER EVALUATIONS (N- 


Utility 

High Medium Low 

1 keJevance ^ 

'potential % % % 

J usefulness ~~ 


1 0 A Reasons for not reading: 

\©/ (N- 

1 Could not readily obtain a copy 
% Not sufficiently Interested 
% Lack of time 
X Other 



Explanatory Note for Exhibit 2A: 

SAMPLE EVALUATION PROFILE PAGE FOR READER AND NON-READER 
DATA — FOR REVIEWS AND PRACTICAL GUIDANCE PAPERS 



A. READER EVALUATIONS 



Means or percentages provided in the first oolurans are -evaluation ratings by 
Readers for that particular document. The second column displays the Reference 
Mean or Percentage for each item, and represents the evaluation ratings for all 
documents within that product-type group (i.e., for Reviews ot Practical Guid- 
ance Papers). Questionnaire items (and response choices where they are not 
given on the form) are provided below. 



1. For your needs, how well did the document cover the topic (s)? 

( ) Poorly ( ) Moderately well ( ) Very veil 

2. Do you feel that the material was up-to-date in its coverage of current 
research or practice, as of its publication date? 

( ) No ( ) not judge ( ) Yes 

3. The organization was: 

( ) less than satisfactory ( ) satisfactory ( ) excellent 

4. The writing was: 

( ) hard to follow at times ( ) moderately clear ( ) very clear 

5. The format (physical layout, illustrations, typography, etc.: 

( ) hindered readability ( ) did not contribute ( ) was very helpful 

and understanding to readability and to readability and 

understanding understanding 

6. The discussion was: 

( ) inadequate for ( ) reasonably ( ) very thoughtful 

my purposes thoughtful 

7. The document was: ( ) About right ( ) Too long ( ) Too short 

8. How relevant was the topic to your general professional interests? 

( ) Not at all relevant ( ) Somewhat relevant ( ) Relevant 

9. As of the publication date, how great was your need for a good document on 
this topic? 

( ) Not at all great; I had no special need for it. 

( ) Moderately great; the topic is of continuing importance to me. 

( ) Very great; I had an immediate need for a document on this topic. 
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(Explanatory Note for 2A cent.) 



10. In general, how wuulh vou compare this document with other documents of t he 
same type? 

( ) Its usefulness is too limited to justify its publication. 

( ) It is not unusually useful, but it is .worth having available. 

( ) It is a very useful document. 

Please use the space provided below for any suggestions you have concerning 
the future preparation of documents of this type, or for elaboration on any 
of your responses above. 

11. Documents can serve a variety of purposes and fulfill many different infor- 

mation needs for readers. In the following question, please indicate how 
useful the document was to you for each of the purposes listed. If you did 
not use the document for a stated purpose, check the last column. 

1'^. As a result of reading the document, did you use the information or the 

document in any of the following ways? 

B. NON-READER EVALUATIONS 



Questionnaire items and response choices (high to low) are provided below. 

1. How relevant do you think this document might be to your general profess- 
ional interests? , n , , 

( ) Relevant ( ) Somewhat relevant ( ) Not at all relevent 

2. As of the publication date, how great was your need for a good document 
of this type, on this topic? 

( )' Document would probably have been very useful. 

( ) Document would probably have" been of some use. 

( ) Document would probably have been of little or no use. 

3. If you knew about the document but did not read or skim it, what reason(s) 

do you remember? 
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EXHIBIT 2B 



© 



SA>1PLE EVALUATION PROFILE PAGE FOR READER AND NON-READER DATA — FOR 
BIBLIOGRAPHIES 

(Document (No.) continued) 



Mder evaluations (N^ 



QUALITY 



Mean 



Reference 

Percentage 



O 



Coverage 

Up*co-d3teness 



0 



[Organ 

Format f ^ ) 
Textual material 



0-- 



( 


.) 


No. oV"t^erences : 


: % 




( 


_) 


About right 






( 


.) 


Too many 






( 


) 


Too few 






( 


) 









UTILITY 



© 

Ptsrpc 



^T^ Relevat 

& 



Mean 



/atjfie 

Need f ® J 

)cooparative usefulness 



Reference Mean 

( ) 



lose of use: 

To identify documents on particular topics 
To identify documents on particular projects 
To identify documents by particular individuals 
To identify documents from particular institutions 
To perform comprehensive search of literature 
To see kinds of new work being reported 



Percentage 

_% 

7 . 

% 

_% 

_% 

% 



Reference 

Percentage 

(_%) 

(_%) 

(_;:) 

i_J) 

( >.) 



0 



IMPACT 



Were cited documents examined? Yes ( %) 



Was content of cited 
document(s) as expected 
from bibliographic 
reference? Yes % 



No 




O 

CD 



N-REAOER EVALUATIONS (N- 



Utility 

Hifih. 

Relevance 



Medium Low 



Potential 
usefulness 



% 



_% 

% 



% 

% 



Q 



Reasons for not reading : 
(N- ) 



Could not readily obtain a copy 
X Not sufficiently interested 
X Lack of time 
X Other 
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’’xplauaLory Note Ku iAiiihit Jb: 



EVALUATION rKOFlLE UAor 
DATA— FOR BIBLIOGRAPHIES 



roR 



i'AnUH AND' NO::-f.rADFK 



A. READER EVALUATIONS 



Means or percentages provided in the first columns are evaluation ratings by 
Readers for that particular document. The second column displays the Reference 
Mean or Percentage for each item, and represents the evaluation ratings for all 
documents within that product-type group (i.e., for Bibliographies). 
Questionnaire items (and response choices where tliey are not given on the 
form) are provided below. 



1. For your needs, how well did the document cover the topic(s)? 

( ) Poorly ( ) Moderately well ( ) Very well 

2. Do you feel that the material was up-to-date in its coverage of current 

research or practice, as of its publication date? 

( ) No ( ) Could not judge ( ) Yes 

3. The classification or organization of entries (references) was: 

( ) less than satisfactory ( ) satisfactory ( ) excellent 

4. The format (physical layout and typography): 

( ) hindered use ( ) did not contribute ( ) was very helpful 

' to its usability to its usability 

5. The textual material (annotations, abstracts, summaries, etc.) was: 

( ) inadequate for my purposes ( ) moderately useful ( ) very useful 

6. The number of references was: ( ) about right ( ) too many ( ) too few 

7. How .relevant was the topic to your general professional interests? 

( ) Not at all relevant ( ) Somewhat relevant ( ) Relevant 

8. As of the publication date, how great was your need for a good document on 
this topic? 

( ) Not at all great; I had no special need for it. 

( ) Moderately great; the topic is of continuing importance to me. 

( ) Very great; I had an immediate need for a document on this topic. 

9. In general, how would you compare this document with other documents of the 
same type? 

( ) Its usefulness is too limited to justify its publication. 

( ) It is not unusually useful, but it is worth having available. 

( ) It is a very useful document. 
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(Explanatorv Note for 2B cont,) 



10. Bibliographies can serve a variety of purposes and fulfill many different 
inforn'iat i on needs for users. In the following question, please indicate 
how vou used the document. (Check as many as apply.) 



11 . 



As a result 
cited? 

1 f yes , was 
the content 



of usiny tlHs docuTTient , did you examine any of the documents 
t ) Yes ( ) No 

the content of the document what you had been led to expect by 
of the bibliographic reference? 

( ) Yes ( ) No 



B . NON -READER EVALUATIONS 



Questionnaire items and response choices (high to low) are provided below: 

1. How relevant do you think this document might be to your general 
professional interests? 

( ) Relevant ( ) Somewhat relevant ( ) Not at all relevant 

2. As of the publication date, how great was your need for a good 
document of this type, on this topic? 

( ) Document would probably have been of little or no use, 

( ) Document would probably have been of some use. 

( ) Document would probably have been very useful. 

3. If you knew about the document but did not use it, what reason (s) do you 

remember? 
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BIT 3. SAMPLE EVALUATION PROFILE PAGE FOR SPECIALISTS’ DATA 

(Document continued) 



SPECIALISTS' EVALUATIONS (N- ) 



o 



QUALITY 





Excellent 


Good 


Fair 


Poor 


Not 

Appl icable 


No 

Response 


Choice of author 














Selection of 

con tent /material 














Choice of references 














Inclusion of current 
material 














Accuracy 














Interpretation 














Organization 














Organization of 
references 














Format 














Writing 















UTILITY 



Would you recommence to colleagues? 



Yes 
If yes: 



No 



© 



Usefulness for Various Purposes 



Purpose "'f Use 



Obtain overview 



Very 

Useful 



Somewhat 

Useful 



Not At 
Al.l Useful 



No 

Response 



Look up facts 



Identify relevant literature 

^ 

Identify individuals or institutions 



Update knowledge 



Obtain new knowledge 



Obtain practical guidance 



Other ; 



Q 



d for Document of This Type 

Very great 
Moderately great 

f 

Not at all great 



0 



Overall Usefulness of Document 
It is a very useful’^document - 

It is not unusually useful, but 

it is worth having available - 

Its usefulness is too limited 

to justify its publication. 
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Explanatory Note for Exhibit 3: 

SAMPLE EVALUATION PROFILE PAGE FOR SPECIALISTS* DATA 



This format reports more completely the response choices for each Item since 
responses were simply tabulated. Means or percentages could not be used 
because of the small number o£ evaluations. Wherever questxons or response 
choices are not clearly incorporated into the form, they are provided below. 



1 Quality. Please rate the quality of the document in each of the following 
areas. If there are any outstanding strengths or weaknesses in an area that 
YOU feel should be considered in the preparation of future documents of this 
type, please use the comments section [after each item] to exp^a n. 

2. Utility. Would you recommend this document to your colleagues in the edu 

rational community? Yes ( ) ' No ( ) 

If yesj fot each of the purposes [given below] Indicate how 
useful you believe it might be. 

3. Need. As of the publication date, how great was the need in the field for 
a g"ood document of this type on this topic? 

( ) Very great; there was an immediate need for a document on this 
( ) Moderately great; the topic is of continuing importance in the field 
( ) Not at all great; there was no special need for it. 

4. Comparative Usefulness . In general, how would you compare this document 
with other documents of the same type? 
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III. INDIVIDUAL DOCKWENT EVALUATION PROFILES 



A. PREP REPORTS 
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Document No. l. Treating Reading Difficulties; The Role of the Principal, Teacher , 

Specialist, Administrator. PREPS 2, 3, 4, 5, Carl B. Smith, et al. , 
1970. (single edition from GPO: OE-30026; separate monographs; 

ED034 078, ED034 079, EP034 080, ED034 081) 



NCEC Product: PREP Report 

Product Type: Practical Guidance Paper 

Subject Cluster: Instructional Content 



Level of Effort Index: High 

Visibility Index: High 



GENERAL FIELD SURVEY (N»170) 

25 % Previously Read/ Skimmed 11 % Only Heard About/Seen 64 % Not Seen/Read 



FAMILIARITY 



RECENCY OF READING 
(N»43) 



9 % Within past month 



21 % Within past 6 months 



21 % Within past 3 months 



49 % More than 6 months ago 



COMMENTS 

READERS: . Instr. . Resources Spec : in responding^ tp__lea^er^s Vj:^ uests ,this 

prep’s most useful. . .generally received favorably by teachers and administrators. 
Other Admin: used in my work in new Title III reading project, jjljpg. Spe^c: used 

to documen^t a term paper. Reading Spec : used during inservice sessions with 

teachers and administrators. Counselor : good references. Prog. Spec : used each 

separate part with appropriate groups. College Admin: has helped to Improve my 

work a great deal. Researcher : documents of this type should be produced 

continually for educator awareness. Instr. Resources Spec : PREP s are excellent 

and timely. 'Reading Spec ; well organized. Prog. Spec : purchased from IRA, not 

from ERIC. College Prof : IRA publication easier to handle and work with. 



SPECIALISTS' SURVEY (N= 



Within past month 
Within past 3 months 



RECENCY OF READING 
(N= 



Cannot recall 
COMMENTS 



Within past 6 months 
More than 6 months ago 



[See Documents lA, IB, 1C, and ID] 



o 

ERIC 
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(Document ^ continued) 



READER EVALUATIONS (N-43) 







QUALITY 




UTILITY 










Reference 






Reference 




Mean 


Mean 




Mean 


Mean 


Coverage 


2.49 


(2.43) 


Relevance 


2.77 


(2.67) 


Up-to-dateness 


2.77 


(2.77) 


Keed 


2.44 


(2^) 


Organization 


2.35 


(2.33) 


Comparative usefulness 


2.63 


(2.52J 


Writing 


2.49 


(2.53) 


Purpose of use: 






Format 


2.63 


(2.74) 


Obtain overview 


2.58 


(1:54) 


Discussion 


2.30 


(2.30) 


Look up facts 


2.23 


(2.24) 






Reference 


Identify individuals 


2.05 


(2.12) 


Percentage 


Percentage 


— ^Identify relevant 


2.21 


■ (2.26) 


Length: 






literature 






About right 


21% 


(83^%) 


Update knowledge 


2.53 


(2.41) 


Too long 


__5% 


CA%) 


Obtain new 


2.09 


(2.18) 








knowledge 






Too shor t 

f : 


_7% 


es%) 









IMPACT 





Percentage 


Reference Percentage 


Used to make decision 


19% 


(«%) 


App^ed in my work 


67% 


(65Z) 


Used to give advice 


^% 


(«Z) 


Examined other dociiments 


16% 


(21*) 


Consulted with author (s) or others 


_5% 


(J%) 


Passed document on to colleague (s) 


56% 


(50%) 



- 


NON-READER 


EVALUATIONS 


(N-108) 


























Utility 










Reasons for not reading: 












(N-19) 




High Medium 


Low 




\ 




Relevance 


62% 


TJX 




53 




Could not readily obtain a copy 


Potential 


55% 


3i0% 


14% 


26 


Not sufficiently Interested 


usefulness 






5 


X 


Lack of time 










11 


X 


Other 

















o 

ERIC 
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Document No. 1A. Treating Reading Difficulties: Reading and the Home EnvlroiuaeuC 

The Principalis Responsibility. 



NCEC Unit: 

p-oduct Type: Level of Effort Index: 

Subject Cluster: Visibility Index: 



GENERAL FIELD SURVEY (N= 

FAMILIARITY 

% Previously Read/Skiramed % Only Heard About/Seen % Not Seen/Read 

RECENCY OF READING 
(N» 

% Within past month % Within past 6 months 

% Within past 3 months % More than 6 months ago 

^ COMMENTS 



SPECIALISTS* SURVEY (N»3) 

RECENCY OF READING 


Within past month Within past 6 months 

Within past 3 months ^ More than 6 months ago 

______ Cannot recall 

COMMENTS 



• Helps to meet principals’ needs for easy-to-read treatments about roles. Well 
organized; comprehensive; solid conclusions; authors well qualified. Includes 
samples of most appropriate references. 

• Did not stick to topic ’’home environment” — much related to usual school programs. 
Useful ideas generated from descriptions of current programs. Topical organization 
of references would be better. 

• Title and contents do not match well. Insufficient treatment of ’’home environ- 
ment.” Reference list incomplete. Page dealing with language of disadvantaged 
children filled with inaccuracies. Little that principal can ’’grab hold of”. 
Organization not systematic. 



er|c 
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SPECIALISTS' EV/^LUATIONS (N=3) 



(Document lA continued) 



QUALITY 





Excellent 


Good 


Fair 


Poor 


Not 

Applicable 


No 

Response 


Choice of author 


1 


2 










Selection of 

content /material 




1 


1 


1 






Choice of references 




2 


1 








Inclusion of current 
material 




1 


2 








Accuracy 




1 


2 








Interpretation 




2 


1 








Organization 




2 




1 






Organization of 
references 




2 








1 


-Format 




3 








- 


Writing 


1 


2 











UTILITY 



Would you recommend to colleagues? Usefulness for Various Purposes 

Yps 1 No 2 ■ — — — 


If yes; 

• 

Purpose of Use 


Very 

Useful 


Somewhat 

Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


1 








Look up facts 




1 






Identify relevant literature 




1 






Identify individuals or institutions 




1 






Update knowledge 


1 








Obtain new knowledge 


1 








Obtain practical guidance 




1 






Other : 



















Need for Document of This Type 



Overall Usefulness of Document 




3 



Very great 
Moderately great 



Not at all great 



2 



It is a very useful document. 

It is not unusually useful, but 
it is worth having available. 

Its usefulness is too limited 
to justify its publication. 
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Document No. IB- Treating Reading Dlff iculf> es: Establishing Central R eading Clinic.*; 

The Administrator's Role. 



NCEC Unit; 

Product Type: Level of Effort Index; 

Subject Cluster; Visibility Index; 



GENERAL FIELD SURVEY (N= 

FAMILIARITY 

% Previously Read/ Skimmed % Only Heard About/Seen % Not Seen/ Read 

RECENCY OF READING 
(N« 

% Within past month ^ Within past 6 months 

% Within past 3 months % More than 6 months ago 

COMMENTS 



SPECIALISTS' SURVEY (N-3) 

RECENCY OF READING ^ 

(N=2) 

Within past month Within past 6 months 

Within past 3 months 2 More thrn 6 months ago 

Cannot recall 

COMMENTS 

• Presents only one side of issue... view traditional and presents little new. Floor 
plans of clinics not helpful. Author somwhat lacking in knowledge of administrators' 
role. Reference list limited .. .several useful references not included. 

•' Bibliography inadequate and incomplete. Could serve as excellent guide for 
administrators setting up clinics. . .very practical. 

• Authors well qualified. References — good selection; not too long. WeiJ. organized. 
Style such that administrators likely to enjoy reading published version. 
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SPECIALISTS' EVALUATIONS (N-'^) 



(Docutaent IB continued) 



QUALITY 





Excellent 


Good 


Fair 

. 


Poor 


Not 

Appl icable 


No 

Response 


Choice of author 


2 












Selection of 

con tent /material 


1 


1 


1 








Choice of references 




1 


1 


1 






Inclusion of current 
material 




2 

1 


1 


U -- 1 

1 






Accuracy 


1 


1 


k 

1 








Interpretation 


1 


1 




1 






Organization 


1 


2 










Organization of 
references 




2 


1 








-Fermat -- 




2 


1 








Writing 


1 


2 











■t 



UTILITY 



Would you recommend to colleagues? 



Usefulness for Various Purposes 



Yes NO 

If yes; 

Cj 

Purpose of Use 


Very 

Useful 


SomewLa t 
Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


2 


1 






Look up facts 




2 


1 




Identify relevant literature 


1 


2 






Identify individuals or institutions 


2 


1 






Update knowledge 


1 


1 


1 




Obtain new knowledge 


1 


2 




_ .. . .. . — - 


Obtain practical guidance 


2 




1 


.. ^ 


Other : 

























Need for Document of This Type 

Very great 

3 Moderately great 
Not at all great 



ERIC 



Overall Usefulness of Document 

2 It is a very useful document. 

1 It is not unusually useful, but 

it is worth having available. 

Its usefulness is too limited 

to justify its publication. 
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Document No. 1C . Treating Reading Difficulties; Correc ting R.-ading Problein s 

in the Classroom. 



NCEC Unit: 

Product Type: 

Subject Cluster: 

GENERAt. FIELD SURVEY (N« 

% Previously Read /Skimmed 



Level of Effort Index 
Visibility Index: 



FAMILIARITY 

% Only Heard About/Seen 



% Not Seen/ Read 



X Within past month 
% Within past 3 months 



RECENCY OF READING 
(N- 



COMMENTS 



% Within past 6 months 
% More than 6 months ago 



SPECIALISTS' SURVEY (N-3) 



Within past month 
Within past 3 months 



RECENCY OF READING 
(N*2) 



Cannot recall 
COMMENTS 



Within past 6 months 

^ More than 6^ months ago 



• Too much reliance on outdated (Strong, Austin), erroneous (Betts) studies. Some 

very obsolete and inaccurate notions of "language." Some excellent, though redundant, 
diagrams. Good emphasis on individualized correction program. Not enough practical 
teaching suggestions^ as implied by title. ^ 

• Very practical and well written. Valid recommendation and conclusions. 

• Quite traditional, added little new to already existing literature. Weakest 
section list of special instructional procedures. .. too brief to be helpful, did not 
tie in to any overall pattern. Topical rather than alphabetic listing of references 
much more useful to teachers. Floor plans of classrooms probably of limited value. 



ERIC 
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(Document 1C continued) 



SP EC I AL 1 ST SJ_ EV ALUAT IONS ( N= '3 ) 



QUALITY 




!! 



TILITY 



Would you recommend to colleagues? 



Usefulness for Various Purposes 



Yes 2 No 1 p 

If yes: 

Purpose of Use 


Very 

Useful 


Somewhat 

Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


1 


1 

2 




— 


Look up facts ' 

Identify relevant literature 


2 








1 THpntifv individuals or institutions 


2 








1 Undate knowledge 


2 








obtain new knowledge 


1 


1 




1 


1 Obtain nractical guidance 


2 








1 Other : 





















O 

ERIC 



Need for Document of This Type 



i Very great 



2 Moderately great 

Not at all great 



Overall Usefulness of Document 
1 It Ifa a very useful document. 

1 It Is not unusually useful, but 

it is worth having available. 

1 Its usefulness is toe limited 
to Justify its publication. 
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Document No. ID- 



Treat iiiR Read in," L- i r i-' i cult les: Treating Readi ng Disabilities . 

The i n 1 i p t ' ? n’ o I 



NCEC Unit: 
Product Type: 
Subject Cluster: 



Level of Effort Index: 
Visibility Index: 



GENERAL FIELD SURV EY (N= 

% Previ.ously Pead/Skiramed 

% Within past month 

% Within past 3 months 



FAMILIARITY 



% Only Heard About /Seen % Not Seen/Read 



RECENCY OF READING 
(N= 



% Within past 6 months 
% More than 6 months ago 



COMMENTS 



i 

i 



SPECIALISTS' SURVEY (N=3) 

Within past month 

Within past 3 months 



RECENCY OF READING 
(N=2) 



Cannot recall 



Within past 6 months 

2 More than 6 months ago 



COMMENTS 

• Well \.rritten; logical treatment; useful. Organization and variety good. Author 
well qualified. 

1 • Lacked adequate research base. Too brief to be of any value to a specialis*.. 

A condensed version of traditional thinking on topic written about in many publications 



• Lacks originality and specificity, 
help for treatment. 



Little attention to "How” ... little practical 



SPECIALISTS' EVALUATIONS (N=3) 



(Document _ continued) 



QUALITY 





Excellent 


Good 


. 

Fair 


Poor 


Not 

Appl icable 


No 

Response 


Choice of author 


1 


2 










Selection of 

cont ent /material 




3 










Choice of references 


1 


2 










Inclusion of current 
material 




1 


2 






< 


Accuracy 




2 


1 








Interpretation 




2 


1 








Organization 




3 










Organization of 
references 




2 


1 








Format 




1 


2 








W'ri ting 




3 











UTILITY 



Would you recommend to colleagues? 



Usefulness for Various Purposes 



I f yes : 

Purpose of Use 


Very 

Useful 


Somewhat 

Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


1 


2 






Look up facts 




2 


1 




Identify relevant literature 


3 








Identify individuals or institutions 


2 


1 J 


1 




Update knowledge 


1 


1 


1 




Obtain new knowledge 


1 




2 




Obtain practical guidance 


1 


2 






Other : 



















N eed ior Document of This 
Very great 



2 Moderately great 

Not at all great 



o •- 

ERLC 



Overall Usefulness of Document 



It is a very useful document . 



It is not unusually useful, but 
it is worth having available . 

Its usefulness is too limited 
to justify its publication. 
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Document No. Bilingual Education, Prep 6 , Horacio Ulibarri, et al., 1969. 

(ED 034 082) 



.NCEC Proauct: PREP Report 

Product Type: Practical Guidance Paper Level of Effort Inaex: High 

Subject Cluster: Special and Other Educational Visibility Index: High 

Groups 

GENERAL FIELD SURVEY (N=106) 

FAMILIARITY 

18 % Previously Read/ Skimmed 15 % Only Heard About/Seen 67 % Not Seen/Read 

RECENCY OF READING 
(N=19) 

21 % Within past month 21 % Within past 6 months 

26 % Within past 3 months 32 % More than 6 months ago 

COMMENTS 

READERS: Researcher ; assisted in the establishment of bilingual reading program. 

Other Admin : need current information of projects across country made available 

on frequent basis. College Admin : not a typical PREP ... others have not been 

useful ... subject needs updating. Prog . Spec : degree of relevance has increased 

enormously since publication. 



SPECIALISTS' SURVEY (N-3) 

RECENCY OF READING 
(N=0) 

Within past month Within past 6 months 

Within past 3 months More than 6 months ago 

Cannot recall 

COMMENTS 

• Very useful for those with little background. Main value clarity. Furnishes basis 
for further investigation. 

• All authors from same institution. . .wider point of view would be helpful. Content 
geared to those with limited knowledge. . .more appropriate for teacher aids... needs 
much teore depth ana breadth. Some excellent resources missing. ■ Put together too 
quickly to be of real value for any time period. Would be mistake to publish as 

new material . 

• Undated; unable to tell who author(s) is. ..messy titling. One sided. ..many 
important points of view left out. Format tacky, jumbled. Too much jargon, 
sentimentality. Should make effort at uniform editions of publications. 



READER EVALUATIONS (N-19) 



(Document 



continued) 



QUALITY 



Reference 





Mean 


Mean 


Coverage 


2.42 


(2.43) 


Up-to-dateness 


2.58 


(2.77) 


Organization 


2.26 


(2.33) 


Writing 


2.47 


(2.53) 


Format 


2.53 


(2.74) 


Discussion 


2.26 


(2.30) 


Percentage 


Reference 

Percentage 


Length: 

About right 


79% 


(ra) 


Too long 


_g% 


(_4%) 


Too short 


21% 


(_82) 



UTILITY 



Relevance 

Need 

Comparative usefulness 

Purpose of use: 

Obtain overview 

Look up facts 

Identify individuals 

Identify relevant 
literature 

Update knowledge 

Obtain new 
knowledge 



IMPACT 



Mean 


Ref erence 
Mean 


2.63 


(2.67) 


2.32 


(2.35) 


2.58 


(2.52) 


2.68 


(2.54) 


2.21 


(2.24) 


2.11 


(2.12) 


2.16 


(2.26) 


2.42 


(2.41) 


2.26 


(2.18) 



Used to make decision 
Applied in my work 
Used to give advice 
Examined other documents 
Consulted with author (s) or others 
Passed document on to colleague(s) 



Percentage 


Reference Percentage 


32% 


(m) 


13% 


(65%) 


3i% 


(ii%) 


26% 


i7J%) 


_5% 


(_6%) 


47% ■ 


(50%) 



NON-READER EVALUATIONS (N- 71) 



Utility 





High 


Medium 


Low 


Relevance 






13% 


Potential 

usefulness 


28% 







Reasons for not reading : 
(N-16) 



25 


% 


Could not readily obtain a copy 


25 


Z 


Not sufficiently interested 


25 


% 


Lack of time 


2Jl. 


X 


Other 
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cont inued ) 



SPECIALISTS' EVALUATIONS (N=3) 



(Document 



QUALITY 





Excellent 


Good 


Fair 


Poor 


No t 

Applicable 


No 

Response 


Choice of author 






1 




2 




Selection of 

con tent /material 


1 




2 








Choice of references 


1 




1 


1 






Inclusion of current 
material 


1 




1 


1 






Accuracy 




1 






1 


1 


Interpretation 


1 








1 


1 


Organization 




1 




1 




1 


Organization of 
references 




1 




1 




1 


Format 




1 


1 






1 


Writing 


1 




1 




0 


1 



UTILITY 



Would you recommend to colleagues? Usefulness for Various Purposes 

Yes 1 No 2 ^ 



If yes: 

Purpose of Use 


Very 

Useful 


Somewha t 
Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


1 








Look up facts • 




1 






Identify relevant literature 


1 








Identify individuals or institutions 


1 








Update knowledge 




1 






• Obtain new knowledge 


1 








Obtain practical. guidance 


1 




' 




Ocher : 



















Need for Document of This Type 



Overall Usefulness of Document 



2 

1 



Very great 
Moderately great 




Not at all great 



It is a very useful document. 

It is not unusually useful, but 
it is worth having available. 

Its usefulness is too limited 
to justify its publication. 



36 



18 



Document No . 3 . Job-Orlented Education Programs for the D isadvantaged. Schools _ ^d 

Industry Cooperate. Prep 9 , Trudy W. Banta, et al., 1969. 

(ED“W4 085) 

NCEC Product: PEEP Report 

Product Type: Practical Guidance Paper Level of Effort Index: 

Subject Cluster: Special and Other Educational Visibility Index. High 

Groups ' - 



GENERAL FIELD SURVEY (N=81) 

FAMILIARITY 

21 % Previously Read/ Skimmed 17 % Only Heard About/Seen _62 — % Not Seen/Read 



29 % Within past 6 months 
47 % More than 6 months ago 



READERS: Unclass : PREP’s should be given wider dissemination. Researcher : 

helped me prepare for working with teachers on the state of the art in teaching 

disadvantaged. 

NON-READERS: Instr. Resources Spec : didn’t realize this title in series was 

available. Researche r ; have only recently developed a need for it. Superintenden t: 
at the time, I believed I was sufficiently familiar with the topic; it now appears 
that I was wrong. 



SPECIALISTS.’ SURVEY (N-3) 

~ * RECENCY OF READING 

(N=0) 

Within past month Within past 6 months 

Within past 3 months Wore than 6 months ago 

Cannot recall 

COMMENTS 

• Abuse of English language. Criteria not given for selection of content... a 
laundry list. 

• Very useful with limited number of clients. Every school should have available. 
Clear communication. . .evaluations carefully drawn. . .recommendations clear and logical. 
Could serve to influence local businesses to inaugurate similar programs. 




6 % Within past month 



18 % Within past 3 months 



RECENCY OF READING 
(N=17) 



COMMENTS 



(Document 3 continued) 



READER EVALUATIONS (N-17) 




QUALITY 




UTILITY 










Reference 






Reference 




Mean 


Mean 




Mean 


Mean 


Coverage 


■ - 2.41 


(2.43) 


Relevance 


2.59 


(2.67) 


Up-to-dateness 


2.82 


(2.77) 


Need 


2.29 


(2.35) 


Organization 


2.29 




Comparative usefulness 


2.82 


(2.52) 


Writing 


2.24 


(2.53) 


Purpose of use: 






Format 


2.59 


(2.74) 


Obtain overview 


2.53 


(2.54) 


Discussion 


2.18 


(2.30) 


Look up facts 


2.41 


(2.24) 






Reference 


Identify individuals 


2.12 


(2.12) 


Percentage 


Percentage 


Identify relevant 


2.29 


(2.26) 


Length: 






literature 






About right 


100% 


(_^%) 


Update knowledge 


2.35 


(2.41) 


Too long 


_0% 


(_4%) 


Obtain new 


2.24 


(2.18) 


Too short 


__g% 


(_8%) 


knowledge 










IMPACT 












Percentage Reference Percentage 


Used to make decision 




24 % 


(^%) 




Applied in my 


work 




n% 


(65%) 




Used to give advice 




53% 


(^%) 




Examined other 


documents 






(m) 




Consulted with author (s) or others 


_6% 


(_6%) 




Passed document on to colleague(s) 


53% 


(^%) 





NON-READER EVALUATIONS (N«50) 


Utility 


Reasons for not reading: 




(jj«14) 


High Medium Low 




Relevance 36% 58% 4% 


21 % Could not readily obtain a copy 


Potential ^8% 52% 18% 


36 % Not sufficiently interested 


usefulness 


21 % Lack of time 


} 


21 % Other 
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(Document _J continued] 



SPECIALISTS' EVALUATIONS (N=3) 



QUALITY 




UTILITY 




Document No. 4. Paraprofesslonal Aides In Education. Prep 12 , Carl H. Rittenhouse, 

1969. (ED 034 906) 



NCEC Product: PREP Report 

Product Type: Practical Guidance Paper Level of Effort Index: iliiill 

Subject Cluster: Educational Administration and Visibility Index: High 

Services 



GENERAL FIELD SURVEY (N=95) 

~ ~ familiarity 

23 % Previously Read /Skimmed 21 % Only Heard About/Seen 56 Not Seen/Read 

RECENCY OF READING 
(N=22) 

18 % Within past month 27 % Within past 6 months 

23 % Within past 3 months 32 % More than 6 months ago 

COMMENTS 

READERS: Prog. Spec : made it available to every school and college in Nebraska. 

Other Admin: used as aid in writing proposal. Researcher : used to provide others 

with information relevant to them. College Prof ; programs of mediocre quality 
included as no truly effective paraprofesslonal training programs developed yet. 
Supervisor : recommended to V.P. for consideration in staffing. 

NON-READERS: Supervisor : just received it. Instr. Resources Spec : did not realize 

existed. Unclass : have referred it to others and rarely have copy for own use. 
Unclass : I like PREP * s. . .would like to see broader distribution of clearinghouse 
products. Other Admin : quite good but not as up-to-date as Education USA 1972 

publication. 



SPECIALISTS* SURVEY (N=3) 

RECENCY OF READING 
(N=l) 

Within past month Within past 6 months 

Within past 3; months More than 6 months ago 

1 Cannot recall 

COMMENTS 



• Good variety. Prpgrams thoughtfully and clearly explained. Tabl,e of contents 
would f^cil^tate usage. Many new federal programs not included. Format — rather 
crowded^-'; no margins on some pages. 

• Very good reference for early systematic efforts in training and utilizing teacher 
aides. 



er|c 









(Document; 



continued) 




QUALITY 



U' 





Mean 


Reference 

Mean 


Coverage 


2.45 


C2.43) 


Up-to-dateness 


2.82 


(IJl) 


Organization 


2.41 


(li21) 


Writing 


2^64 


(2.53) 


Format 


2.86 


(2.74) 


Discussion 


2.27 


(2.30) 



pf icentage 



Length: 

About right 95 % 
Too long 0 % 
Too short Q % 



Reference 

Percentage 



(^%) 
( 8 %) 





Mean 


Ref erence 
Mean 


Relev nee 


2.68 


(2.67) 


Need 


2_.J5 


(2.35^ 


Comparative usefulness 


2.55 


(2.52) 


Purpose of use: 


Obtain overview 


2.73 


(2.54) 


Look up facts 


2.27 


(2.24) 


Identify Individuals 


2.09 


(2.12) 


Identify relevant 


2.23 


(2.26) 


literature 


Update knowledge 


2.41 


(2.41) 


Obtain^ew 


2.36 


(2.18) 



IMPACT 



Used to make decision 
Applied in my work 
Used to give advice 
Examined other documents 
Consulted with author (s) or others 
Passed document on to colleague(s) 



Percentage 

55% 

23% 

_5% 

55% 



Reference Percentage 

( m) 

( ^%) 

( m) 

( 27%) 

( 50%) 




ERIC 



SPECIALISTS' EVALUATIONS (N=3) 



(Document 4 continued) 



QUALITY 





Excellent 


Good 


Fair 


Poor 


Not 

Applicable 


No 

Response 


Choice of author 


1 


1 










Selection of 

con tent /mater ial 


3 












Choice of references 


1 


O 

iU 






> 




Inclusion of current 
material 


1 


1 


1 








Accuracy 


3 












Interpretation 


1 


2 










Organization 


1 


1 


1 








Organization of 
References 




2 






1 




Format 




1 


2 








Writing 


2 


1 











UTILITY 



Would you recommend to colleagues? 



Usefulness for Various Purposes 



If yes: 

Purpose of Use 


Very 

Useful 


Somewhat 

Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


3 








Look up facts 


2 


1 






Identify relevant literature 




3 






Identify individuals or institutions 


1 


1 


1 




Update knowledge 


1 


2 






Obtain new knowledge 


1 


2 






Obtain practical guidance 


2 


1 






Other: '■* 












s 







Nee d for Document of This Type 

3 Very, great 
_____ Moderately great 
Not at all gfeat 



•• 



O L. 

ERIC 



J 



Overall Usefulness of Document 

J It is a very useful document. 

It is not unusually useful, but 

it is worth having available. 

Its usefulness is too limited 

to justify its publication. 
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Document No. 5 s haring Educat ln nal Services- Pxep Ray Jongeward and 

Fran Heesacker , 1969. (ED 036 666) 



PiouacL; PKIP Report 



Product Type: Practical Guidance Paper Level ot Effort Index: 

Subject Cluster: Special and Other Edu cat tonal Vlslbllltv Index: 

Groups 



GENERAL FIELD SURV EY (N= 1 07 ) 
15 % Previously Read/Skimmed 

19 % Within past month 
25 % Within past 3 months 



FAMILIARITY 

21 % Only Heard 

RECENCY OF READING 
(N=16> 



COMMENTS 



About /Seen Z Not Seen/ Read 

31 % Within past 6 months 

25 % More than 6 months -go 



READERS: Researcher: does not address itself to the most significant problem 

involved. Principal: used it for in_service staff training. Prog .j Sj^ec : used in 

writing a proposal . ..PREP is good publication. Researcher : difficut to obtain. 

NON-READERS: Other Admin: assigned to other staff members. Researcher : too thick- 

Researcher : talked directly with principal researchers. Researcher : publicized 

its existence. 



SPECIALISTS' SURVEY (N=3) 



Within past month 
Within past 3 months 

COMMENTS 



RECENCY OF READING 
(N=0) 



Cannot recall 



Within past 6 months 
More than 6 months ago 



• Well written, factually presented. Very helpful to educational institutions. 
Facts well interpreted. 

• Would like to have known dates of implementation of each project. So brief, 
it is most useful as point of reference only, or for identification purposes. 
Interested in more information such as feasibility of projects. 

• Would like to have seen more information sheets by Dr. Jongeward. NWREL has 
done a number of outstanding pro j ects. .. this is another in keeping with that reputa- 
tion. 



N 
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{Documenl 3 



cont inued ) 



READER EVALUATIONS (N=16) 




QUALITY 




UTILITY 










Ref erencc 






Ref erence 




Mean 


Mean 




Mean 


Mean 


Coverage 


2.44 


(2v^3) 


Relevance 


2_.7_5 


(2^*A7) 


Up-to-dateness 


2 .88 


(2.77) 


Need 


2.19 


(2.35) 


Organization 


2.19 


(2.33) 


Comparative usefulness 


2.73 


(1.^52) 


Wri' V- 


2.56 


(2.53) 


Purpose of use: 






Format 


2.81 




Obtain overview 


2.75 


(lr_52) 


Discussion 


2.31 


(2_-A0) 


Look up facts 


2.06 


(2.24) 






Reference 


Identify individuals 


1 . 06 


(2.12) 


Percentage 


Percentage 


Identify relevant 


2.38 


(2.26) 


Length : 






literature 






About right 


81% 


(m) 


j Update knowledge 


2.63 


(2.A1^ 


Too long 


_6% 


(^%) 


Obtain new 


2.31 


(2.18) 


Too short 


_6% 


(_8%) 


knowledge 










IMPACT 












Percentage Reference Percentage 


Used to make decision 




25% 


(23%) 




Applied in ray work 






(65%) 




Used to give advice 






(49%) 




Examined other 


documents 




21% 


^27%) 




Consulted with 


author (s) or others 


21% 


(^%) 




1 L ocuraent 


: on to colleague(s) 
— 


69% 


(^%) 





NON-READER EVALUATIONS (N=69) 


Utility 


Reasons for not reading: 




(N=22) 


High Medium Low 




Relevance 28% 52% 20% 


23 % Could not readily obtam a copy 


Potential ^gc, jq./ 


27 % Not sufficiently incere«t“U 


usefult.e.ss — 


14 % Lack of time 


■ 


32 % Other 
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’(Document 



cnnt inued ) 



SPECIALISTS’ EVALU_AJ10NS (N=3) 



QUALITY 



Writing 




Cho i c t' f iiuthor 

Selection of 

con tent /ma terial 

Choice of r eferences 

Inclusion of current | 
materia l 

Accuracy 

Interp retati on 

Organization 

Organization of 
references 

Format 



UTILITY 




Would you recommend to colleagues? 
7 - No 



Usefulness for Various Purposes 




Docufnen t No . 6 . Social Studi es _and 

Jonathon C. McLendon, et al • , 1970. {ED 037 ^oo) 



NCEC Product: PREP Report 

Product Type: Practical Guidance _Pa 2 er 

Subject Cluster: Instructional Content 






Level of Effort Index: High 
Visibility Index: Hifih 



GENERAL FIELD SUR VEY (N= 63) 

16 % Previously Read/ Skimmed 

20 % Within past month 
20 % Within past 3 months 



FAMILIARITY 

13 % Only Heard About /Seen 



71 % Not Seen/ Read 



RECENCY OF READING 

(N= 10) 



COMMENTS 



40 % Within past 6 months 
20 % More than 6 months ago 



READERS: Pr og . Spec : made available to every school in state. 

NON-READERS: Superintendent : forwarded to Department Head. 



ERIC 



SPECIALISTS' SURVEY (N=2) 



Within past month 
Within past 3 months 



RECEN CY OF R EADING 
(N=0) 



Cannot recall 
.COMMENTS 



Within past 6 months 
More than 6 months ago 



• Author respected as social studies educator- Excellent, relevant topic. More 
specific footnotes needed. Lacks bibliography ... author ' s name and date do not 
constitute a useful reference. Section introductions present key questions or outline 
major ideas to be covered — helps in reading a very unattrative manuscript. Many 
paragraphs too long. Broad research, clear conclusions, specific recommendations 

most valuable aspects of paper. 

• Lack of detailed, prefer rably annotated bibliography a serious failing. Writing 
clear but heavy and laborious. Content basically good and very timely. Paper falls 
down primarily because it is sometimes unnecessarily difficult to read (because of 
typography or language), sections not well coordinated, and some generalizations an 
implications not well supported. Latter not entirely author’s fault since scope xs 
broad and available research scant. 



I 8 
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(Document b continued) 



READER EVALUATIONS (N= 



QUALITY 


*• UTILITY 


Reference 


Reference 


Kean Mean 


Mean Mean 


Coverage 


Relevance 


Up-to-dateness 


Need 


Organization 


Comparative usefulness 


Writing 


Purpose of use: 


Format 


Obtain overview 


Discussion 

• < 


Look up facts 


Reference 


Identify individuals 


Percentage Percentage 


Identify relevant 


Length: 


' ' literature 


About right 


‘Update knowledge 


Too long 


Obtain new 




knowledge 


Too short 

% 





IMPACT 

Percentage Reference Percentage 

Used to make decision 
Applied in my work 
Used to give advice 
Examined other documents 
Consulted with author (s) or others 
Passed document on to colleague(s) 



NON-READER EVALUATIONS (N=45) 




Utility 


Reasons for not reading: 




(N»8) 


High Medium Low 




Relevance 38% 11% 


13 % Could not readily obtain a copy 


Potential „ , , 


38 X Not sufficiently interested 


^ , 33% 44% 22% 




usefulness — — — 


38 % Lack of time 


f 


13 %■ Other 
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(Document 6 continued) 



SPECI ALISTS' EVAUJATIQN S (N=2) 

QUALITY 





Excellent 


Good 


Fair 


Poor 


Not 

Appl icable 


No 

Response 


Choice of author 




2 










Selection of 

content /material 


1 








1 


^ 


Choice of references 




1 




1 






Inclusion of current 
material 


1 


1 










Accuracy 




2 










Interpretation 


1 


1 










Organization 


1 




1 








Organization of 
references 






2 








Format 






1 


1 






Writing 






L ._i_J 







UTILITY 



Would you recommend to colleagues? Usefulness for Various Purposes 

9 No 


If yes; 

Purpose of Use 


Very 

Useful 


Somewhat 

Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


2 








Look up facts 




2 






Identify relevant literature 


1 




1 




Identify individuals or' institutions 


2 








Update knowledge _ 


2 








Obtain new knowledge 


2 








Obtain practical guidance 


2 








Other : 

























er|c 



Need for Document of This Type 

2 Very great 

Moderately great 

Not at all great 



Overall Usefulness of Document 
_2 It is a very useful document. 



It is not unusually useful, but 
it is worth having available. 

Its usefulness is too limited 
to justify its publication. 



IF 



% 
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Document No. 7 



Ind ividualized Ins ^’rep 16 . Jack V. kdling, 19 70. 

(¥D’lT4T~ld3) 



NCtiC Product: PRHP Report 

Product Type: Practical Guidance Paper 

Subject Cluster: Educational Administration 

and Services 



Level of Effort Index: High 

Visibility Index: High 



GENERAL FIELD SURVEY (N-251) 



FAMILIARITY 



30 % Previously Read/ Skimmed lA % Only Heard About /Seen Seen/Read 

RECENCY OF READING 



(N-76) 



21 % Within past month 
24 % Within past 3 months 



22 % Within past 6 months 
33 % More than 6 months ago 



COMMENTS 

READERS: Superintendent : excellent. Other AdnAn^ ? catalyst for others. . .are 

individualizing all areas in our school. Prog. Spec : made available to e^ery 

school system in Nebraska. Supervisor : used in workshop. . .helped provide informa- 
tion for setting priorities. Instr. Resources Spec : more copies should be given 

to states. . .saved digging. Sec. Teacher : would like to see something similar for 
classroom teachers ... gave support in what we are doing. Sec. used in 

workshop for teachers. Researcher : tried to use it and couldn t... wanted 

theoretical perspective. Elem. Teacher ; more concise report needed. Frog. Sp_ec : 
like clear subdivisions. . .increased usefulness. NON- READERS : Oth^r Admin , more 

up-to-date documents are available. Researcher : PREP s don t give enough 

information. 



SPECIALISTS* SURVEY (N=2) 



Within past month 
Within past 3 months 



RECENCY OF READING 
(N=0) 



Cannot recall 
COMMENTS 



Within past 6 months 
More than 6 months ago 



• Well organized; well thought out. A little confusing format, maybe 
because a composite report. For school person interested in individualized 
instruction, document succeeds in meeting goals. 

• Disorganized, sloppy writing; format bad. Unclear what project is about until 
well along in text. Only useful to pinpoint places to look for individualized 
instruction ref erences ... could be done in 5 or 10 pages and veil summarized. 






O 

ERIC 
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(Document 7 



continued) 



READER EVALUATIONS (N=76) 












QUALITY 




UTILITY 










Reference 






Reference 




Mean 


Mean 




Mean 


Mean 


Coverage 


2.51 


(2.43) 


Relevance 


2.76 


(2.67) 


Up-to-dateness 


2.82 


(2.77) 


Need 


2.51 


(2.35) 


Organization 


2.37 


(2.33) 


Comparative usefulness 


2.59 


(2.52) 


Writing 


2.47 


(2.53) 


Purpose of use: 






Format 


2.74 


(2.74) 


Obtain overview 


2.63 


(2.54) 


Discussion 


2.34 


(2.30) 


Look up facts 


2.25 


(2 . 24 ) 






Reference 


Identify individuals 


2.09 


(2.12) 


Percentage 


Percentage 


Identify relevant 


2.30 


(2.26) 


Length: 






literature 






About right 




(83%) 


Update knowledge 


2.49 


(Ml) 


Too long 


_7% 


CA%) 


Obtain new 


2.26 


(2.18) 


> Too short 


__5% 


C_8%) 


knowledge 








IMPACT 












Percentage Reference Percentage 


used to make decision 






(23%) 




Applied in my 


work 






(^%) 




Used to give advice 






(^%) 




Examined other 


documents 




36% 


(m) 




Consulted with author (s) or others 


J% 


(_6%) 




I Passed document on to colleague (s) 


58^ 


(^%) 





NON-READER EVALUATIONS (N-139) 



Utility 





High 


Med ium 


Low 


Relevance 


63% 


30% 


7% 


Potential 

usefulness 


57% 


30% 


13% 



Reasons for not reading : 

(N“36) 

42 % Could not readily obtain a copy 

22 % Not sufficiently Interested 

8 Lack of time 
22 %. Other 
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SPECIALISTS' EVALUATIONS (N-z) 



(Document 7 continued) 



QUALITY 





Excellent 


Good 


Fair 


Pool 


Not 

Applicable 


No 

Response 


Choice of author 








1 


1 




Selection of 

content/ ma ter ial 


1 


1 - 










Choice of references 




2 










Inclusion of current 
material 




2 










Accuracy 


1 




1 








Interpretation 


1 






1 






Organization 


1 






1 






Organization of 
references 




1 






1 




Format 






1 






1 


Writing 


1 




1 












UTILITY 



Would you recommend to colleagues? 



Usefulness for Various Purposes 



If yes: 

Purpose of Use 


Very 

Useful 


Somewhat 

Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


1 








Look up facts i 


1 








Identify relevalnt literature 


1 








Identify individuals or institutions 


1 






* 


Update knowledge 


1 








Obtain new knowledge 


1 








Obtain practical guidance 


1 








Other: 



















Need for Document of This Type 

2 Very great 

Moderately great 

Not at all great 
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Overall Usefulness of Document 

1 It is a very useful document. 

It is not unusually useful, but 

it is worth having available. 

_1 Its usefulness is too limited 

to justify its publication. 

~TT 



B. EMC BIBLIOGRAPHIC 



O 

ERIC 



DoCUfTtent No. Books Related to English Language and Lij:.erature In Elementary 


and Secondary Schools, Lois B. 
Stanley (Comp.), October 1969. 


Watt, Delia Goetz, and Caroline 
(GPO: OE-30024; ED 039 236) 


NCEC Product: EMC Bibliography 

Product Type: Bibliography 

Subject Cluster: Instructional Content 


Level -f Effort Index: High 

Visioility Index: Medium 


GENERAL FIELD SURVEY (N= 397) 

FAMILIARITY 



5 % Previously Read /Skimmed ^3 % Only Heard About/Seen Not Seen/ Rend 



RECENCY OF READING 


0 % Within past month 
22 % Within past 3 months 

COMMENTS 

READERS: Reading Spec: provided stimulus and interest as well as knowledge enabling 

teacher to utilize creative writing with remedial reading youngsters. Principal : 
find it particularly helpful in helping give guidance to specific departments on 
curriculum improvement. College Prof : was limited in providing description I needed. 

NON-READERS: Instr. Resources .Spec ; have taken a reference from it. Instr . 

Resources Spec : have our own buying glide. ..this used for information. Supervisor : 

at time, was not involved in my present position. Prog; ijSpec ; would need to be 
updated constantly to be of service for textbook adoptions. Prog. Spec ; as a 
state consultant, did use for reference purposes. Instr. Resour ce s Spec : would help 

to include LC number in Trade Books — Juvenile Literature — section for ordering 
purposes. Prog. Spec : review material only, bupervxsor : impression tnat not detail- 
ed or analytical enough to be of much help. 



SPECIALISTS' SURVEY (N=3) 

RECENCY OF READING 
(N=0) 

Within past month Within past 6 months 

Within past 3 months More than 6 months ago 

Cannot recall 

COMMENTS 

• Material very incomplete; many worthwhile texts not included. Trade book.? dealing 
with minority groups not adequately represented. Too many o»nissions to be considered 
comprehensive. .. too broad a range to be considered specialized. 

• Too incomplete and unrepresentative to be very useful. If this is all of the 
books and materials EMC received during Jan. 1968 to Oct. 1969, they need better 
detectives as scouts. 

• I have never read Anything by tnese authors in the professional literature. 
Noticeable omissions of excellent language arts text books. Since based on EM.C 
acquisitions , possible that better resources are not listed because publishers failed 
to send them? No excuse for very traditional topics under which language arts and 
reading texts are licted# ,» forces omission of sources resulting from efforts to 
implement recommendations of the Dartmouth conference. Too much in one publication... 
one for elementary school resources and another with secondary far more realistic. 
Neither timely enough nor comprehensive enough to be particularly useful to curriculum 
people or for book selection. 





% Within past 6 months 
% More than 6 months ago 



(Document 8 continued) 



READER EVALUATIONS (N> 



Coverage 

Up-to-dateness 

Organization 

Format 

Textual material 



QUALITY 

Mean Rei .. cnee Mean Percentage 

^o. of rererences: 

About right 
Too many 
Too few 



Reference 

Percentage 



UTILITY 



Mean 



Reference Mean 



Relevance 

Need 

Cc mparative usefulness 

Purpose of use: 

To identify documents on particular topics 
To identify documents on particular projects 
To identify documents by particular individuals 
To identify documents from particular institutions 
To perform comprehensive search of literature 
To see kinds of new work being reported 



Reference 

Percentage Percentage 



IMPACT 



Were cited documents examined? Yes 



Was content of cited 
document (s) as expected 
from bibliographic 
reference? Yes % 



No 



NON-READER EVALUATIONS (N-163) 
Utility 



Reasons for not reading 
(N-2’ 





High 


Medium 


Low 








Relevance 




36% 


18% 


28 


% 


Could not readily obtain a copy 


Potential 


34% 


42% 


23% 


16 


Z 


Not sufficiently interested 


usefulness 


0 


% 


Lack of time 



52 i other 



o 
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(Document 8 



cont i nued ) 



SPECIALISTS’ EVALUATIONS (N=i) 



QUALITY 





Excellent 


Good 


Fair 


Poor 


Not 

Appl ic able 


No 

Re spons V 


Choice of author 






1 


1 




1 


Selection of 

content /material 






1 


] 


1 




Choice of references 




1 


1 






1 


Inclusion of current 
material 






1 


7 






Accuracy 


1 


1 






1 




Interpretation ^ 


1 






1 


1 




Organization 






1 


1 


I 




Organization of 
references 




3 










Format 




3 










Writing . 

m 


j 1 


2 




1 







UTILITY 



Would you recomniend to colleagues? 
Yes No 3 



Usefulness for Various Purposes 



If yes: 

Purpose of Use 


Very 

Useful 


Somewhat 

Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 










,-Look up facts 










Identify relevant literature 










Identify individuals or institutions 










Update knowledge 










Obtain new knowledge 










Obtain practical guidance 










Other : 



















Need for Document of This Type 

1_ Very great 

1 Moderately great 
1 Not at all great 



Overall Usefulness of Document 

It is a very useful document . 

2 It is not unusually useful, but 

it is worth having available. 

I Its usefulness is too limited 

to justify its publication. 
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Document No 



S c i e no 0 and Ma the-i T,.it ics Bo c ks for Eletnenrarv and Seconda ry Schoo 1 s , 
Lois B. Watt, Della Goetz, and Eunice von Ende (Comp.)» February 
1970. (GPO: OE-29071;ED OAl 760) 



NCKC Product: KMC Bibliography 

P r 0 d u c t Type: B ib liogra phy 

Subject Cluster: Instruc tional Content 



Level of Effort Index: 
Visibility Index: Medium 



GENERAL FIELD SURVEY ( 182 ) 



7 __ % Pteviousiy Read/Skitnmed 

8 % '-lithin past itonth 

8 % Within past 3 months 



F AMILIARITY 

13 ^ Only Heard About/ Seeh % Not Seen/Read 

RECENCY OF READING 

% Within past 6 months 
% More than 6 months ago 



COMMENTS 



NON-RE/.DERS : Prog. Spe c; unaware of content material breakdown, 

great. Rese a rcher : budget limitations. Instr. Resources Spec : 

atte’itlon . 



Unclass : sounds 

just came to my 



SPECIALISTS' SURVEY (N=3) 



VJithin past month 
Withi.n past 3 months 



RECENCY OF READING 
(N=l) 



Cannot recall 
COMMENTS 



1 Within past 6 months 

More than 6 months ago 



• Well organized. Do not know compilers — assume they are well qualified for the job. 

• Very limited coverage. A bibliography of detailed bibliographies in these 
fields, containing reliable analytical reviews would be more worthwhile. 

• Some re-organization might have made bibliography more useful — "General Science" 
section a catch-all; several books in "History" section were not science histories. 
"Biology" section might have included subsection on microbiology etc. Could be 
longer . 



28 



57 



(D.Hument 



c .nu inued) 



R'£ADER EVALUATIONS (N= 



Mean 



» •' erage 

j Up-to-dateness 
Organization 
Format 

Textual material 



QUALITY 

Reference Mean Percentage 

No. of references: 

About right 
Too many 
Too f ew 



Ref erence 
Percentage 



UTILITY 



Mean 



Reference Mean 



Relevance 

Need 

Comparative usefulness 

Purpose of use: 

To identify documents on particular topics 
To identify documents on particular projects 
To identify documents by particular individuals 
To identify documents from particular Institutions 
To perform comprehensive search of literature 
To see kinds of new work being reported 



Percentage 



Reference 

Percentage 



IMPACT 



Were cited documents examined? Yes 



Was content of cited 
document (s) as expected 
from bibliographic 
reference? Yes % 



No 



er|c 



NON-READER EVALUATIONS (N-146) 

Utility 



Relevance 

Potential 

usefulness 



Realms for not reading : 
(N»24) 



High 


Medium 


Low 










38% 


16% 


25 


% 


Could noj: readily obtain a copy 








29 


% 


Not sufficiently interested 


34% 


28% 


23% 














13 


•a 


Lack of time 








V 29 


% 


Other 
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bPtClALlSTS' EVALUATIONS (N- 



(Document 



cont inued ) 



QUALITY 




UTILITY 



Would you recommend to colleagues? 

Yes 2 No T 

T f yen : 

Purpose of Use 



Usefulness for Various Purposes 



Obtain overview 



Look up facts 
Identify relevant literature 



Very 

Useful 



Somewhat 

Useful 



Not At 
All Useful 



No 

Response 




Need for Document of This Type 

1 Very great 

2 Moderately great 

Not at all great 



O 

ERIC 



Overall Usefulness of Document 
1 It is a very useful document. 
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1 It is not unusually useful, but 
it is worth having available. 

1 Its usefulness is too limited 
to justify its publication. 






Document No. lO* * Books Related to Adult Baalc Education and Te aching English to 

Speaker s of Ot.h ar Languages ♦ Myra H. Thomas, Thelma M. Knuths, Sid“ 
ney E. Tlurphy (I'.orap.), May 1970. (CPOt OE 13039; ED 043 850) 



NCEC Product: EMC iiibiiography 

Product Type: Bibliography Level of Effort Index: Medium 

Subject Cluster: Special and Other Educational Visibility Index: Medium 

Groups 



GENERAL FIELD SURVEY (N- 174) 

FAMILIARITY 

S % Previously Read/Skimmed 20 Only Heard About/Seen '7 /<> Not Seen/Read 

RECENCY OF READING 
(N» 14) 

7 % Within past month Within past 6 months 

% Within past 3 months 50 % More than 6 months ago 

COMMENTS 

PKaDERS: College Prof : perforas useful function in light of subject area 

limitations. Instr . Resources- 'Spec : poor and misleading document .. .EMC did not 

have collection to support purpose of document ... it was an inappropriate agency 
to produce it. Prog. Spec: needs more emphasis on student materials. Prog. Sp^: 

more depth needed in abstracts. Col lege Prof : add critical book review type comments 

to bibliography. Prog, spec : Uue in my role as instructional resource consultant. 

NON-READERS: Prog. Spec : not presently doing work in area but am personally 

interested. Researcher : scanned for relevance for individual projects. 



SPECIALISTS’ SURVEY (N-3) 

“ RECENCY OF READING 

(N-0) ; ^ 

Within past month 1 Within past 6 months 

Within past 3 months More than 6 months ago 

Cannot recall 

COMMENTS 

• Glad to lear’-* about this resource — not aware of its existence. This reconfirms 
my opinion that ERIC has to assume various dissemination roles. There should be 
some interpretation (e.g., evaluation, identification of audience for whom the 
resource was prepared, or statement of purpose). Exclusion of evaluative judgements 
by qualified persons is serious limitation. 

• Clear categories; well organized; easy to read and understand. Would be more 
useful if it were an annotated bibliography. Should be published regularly (bi- 
annually or annually) to keep up-to-date. 

• Inclusion of more available student materials would have increased value of 
document. Materials well organized in a very useable fashion. 
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(Document IQ continued) 



READER EVALUATIONS (N= 



Mean 



QUALIl’Y 

Reference Mean 



Percentage 



Reference 

Percentage 



Coverage 

Up-to-dateness 

Organization 

Format 

Textual material 



No. of references: 
About right 
Too many 
Too few 



UTILITY 



Mean 



Reference Mean 



Relevance 
Need 

Comparative usefulness 

Purpose of use: 

To identify documents on particular topics 
To identify documents on particular projects 
To identify documents by particular individuals 
To identify documents from particular institutions 
To perform comprehensive search of literature 
To see kinds of new work being reported 



Reference 

Percentage Percentage 



IMPACT 



Were cited documents examined? Yes 



Was content of cited 
document (s) as expected 
from bibliographic 
reference? Yes % 



No 



% 



NON-READER EVALUATIONS (N-125) 
Utility 



Potential 

usefulness 



Reasons for not reading : 
(N-35) 



High 


Medium 


Low 








37% 


29% 


34% 


34 


% 


Could not readily obtain a copy 








17 


% 


Not sufficiently Interested 


3 


% 


Lack of time 








37 


% 


Other 



o 

ERIC 



c2 
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(Document 1 0 



cone inued ) 



SPECIALISTS 



EVAU'AT’ 



IONS 



^ i 



QUALITY 





Excellent 


Good 


Fail 


Poor 


Not 

Applicable 


No 

Kesponso 


r- - - • • ' 

Cf'oic e oi authu’.' 


1 


1 








1 


Selection oi 

content /material 


1 


1 






1 




Choice of references 


1 


1 






1 




Inclusion of current 
material 


i 3 


1 


• 








Accuracy 


3 












Interpretat ion 








1 


2 




Organ! ration 


1 








2 




Organization of 
references 


3 












j 

Format 


3 












Writing 


1 


1 






1 





UTILITY 



Would you recommend to colleagues? 



Usefulness for Various Purposes 



If yes: 

Purpose of Use 


Very 

Useful 


Somewha t 
Useful 


Not At 
All Useful 


No 

Response 


Obtain overview 


1 


1 


1 




Look up facts 


2 




1 




Identify relevant literature 


3 








Identify individuals or institutions 


1 


2 






Update knowledge 


2 


1 






Obtain new knowledge 


1 


1 


1 




Obtain practical guidance 


1 


1 


1 




Other : 



















Need for Document of This Type 

3 Very great 

Moderately great 

Not at all great 



Overall Usefulness of Document 

! It is a very useful document. 

It is not unusually useful, but 

it is worth having available. 

Its usefulness is too limited 

to justify its publication. 

T3 
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DocuFnent No, 



I v’’f uli c P I' o f ~ s s i o n , I' ii n i 

JiTly 1970^ (Gi^~r^E~ldbbO-A; ED 04b 890) 



L' 'HI '.I Fn(1i’’ 



( ('limp . ) , 



Nui.C 



I ' V ‘'I *1 ^ i— ( * 



V'.\C lUbl iop,r.'iP^'y 



Product Tvpc: Bi bliogra phy 

Sublert Cluster: E ducational A droXalstratlon and 

Services 

GENERAL FIELD SURVEY (N= 2b0) 

- — - familiarity 

9 % previously Read/ Skimmed JA % Only Heard 



Level of Effort Index: Hl^h 

Visibility Index: Med iura 



About /Seen 72 Not Seen/ Read 



0 1 Within past month 

4 % Within past 3 months 



RECENCY OF Rl/VDING 

(N=23) 


39 


% Within past 


6 months 




57 


% More than 6 


months ago 


COMMENTS 









READERS: Instr. Re sources Spec : author index or combined analytical index would 

add to usefulness. Instr. Resources Spec : not particularly comprehensive for 

time period or for sub j ect ... have not found helpful. Other Admin ; need annotations 
or reviews with annotations. Instr. Resources Spec : need annotations or scope 

notes ... format is acceptable. Instr. Resources Spec : used to build teacher s 

professional collection. 



NON-READERS: Prog. Spec : have many similar publications in library. lnst _r . 

Resources Spec ; our service tries to put users in touch with more easily accessible 
materials. College Prof ; only browsing at time. College Prof : keep as general 

ref ercnce . 



SPECIALISTS* SURVEY (N»3) 

Within past month 

Within past 3 months 



RECENCT OF READING 
(N=0) 



Cannot recall 
COMMENTS 



Within past 6 months 
More than 6 months ago 



• Very valuable bibliography; very complete. Good organization. 

• Well done. 

• Almost impossible to evaluate — no statements of purpose intended audience j how 
books were acquired or criteria used to select items for entry. Any bibliography 
is useful (and this one is too) particularly when well organized, but has limited 
usefulness without short descriptions of entries. 
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(DtnumeiU ^ 1 



. uuL i nueJ ) 



0 

evaluations 


(N=J j) 


QUALm 




Ref et enre 




Mean 


Reieience Mean 


Percentage 


Percentage 


Coveiage 


-.‘JA 


U.h9) 


No. reierences: 




Up-to-dateness 




(2.78) 


About right !> 77 . 


(81 4 ) 


Organization 


2_.2b 


(2^23) 


Too many 94 


( 44) 


Format 


-.zJA 


i2_.T2) 


Too few 224 


( U 4 ) 


Textual material 


2.2b 


( 2 . 47 ) . 







UTILI-n' 







Mean Reference Mean 






Relevance 


2.78 








Need 


2 ■ 30 


(2.39) 






Comparative usefulness 


2.65 


(2.70) 


Reference 


Purpose 


of use: 


Percentage 


Percentage 


To 


identify documents on particular 


topics 




(7_1^) 


To 


identify documents on particular 


projects 




(41%) 


To 


identify documents by particular 


individuals 


_0% 


(13%) 


To 


identify documents from particular institutions 


_9% 


(11%) 


To 


perform comprehensive search of 


literature 


39% 


(31%) 


To 


see kinds of new work being reported 


61% 


(IZ.%) 




IMPACT 


•m 




Were cited documents examined? Yes 2^ 


(87%) Was content of cited 

document (s) as expected 
from bibliographic 
reference? Yes 83 


X No 17 % 



NO N>READ£R EVALUATIONS (N-200) 

Utility 



Relevance 

Potential 

usefulness 



Reasons for not reading : 
(N-37) 



High 


Medium 


Low 










35% 


13% 


41 


X 


Could not readily obtain a copy 


39% 


39% 




27 


X 


Not sufficiently interested 








8 


X 


Lack of time 



19 % Other 



o 

ERIC 



c o 
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(DCv'ument n 



i" t 'I n t i I'i Li t' vi ) 



SPECIAL ISIS' EVAlDATIONS 

QUALITY 





Excellent 


Good 


Fair 


Pi'ii'r 


Not 

A p P 1 ‘ li b 1 l.' 


No 

P.^ spv'n.'^t 


Llioii c oi liuLhor 




1 ^ 






1 


. 


Selection of 

con tent /material 


1 










Choice of references 


2 




1 








Inclusion of current 
material 


3 












Accuracy 


1 








2 




Interpretat ion 










3 




Organization 


1 






1 


1 




Organization of 
references 


1 


1 


1 




!>■ F ■ 




Format 


2 


1 










Writing 








1 


7 





UTILITY 



Would you recommend to colleagues? 



Usefulness for Various Purposes 



If yes: ^ j 

Purpose of Use 


Very 

Useful 


Somewhat 

Useful 


Not At 
All Usoful 


No 

Response 


Obtain overview 


1 


1 


1 




Look up facts 


1 


1 


1 




Identify relevant literature 


2 


1 






Identify individuals or institutions 


1 


2 






Update knowledge 


2 


1 






Obtain new knowledge 


1 




2 




Obtain practical guidance 


1 




2 




Other : 



















Need for Document of This Type 

2 Very great 
1 Moderately great 
Not at all great 



Overall Usefulness of Document 
L It is a very useful document. 



It Is not unusually useful, but 
it is worth having available. 

Its usefulness is too limited 
to justify its publication. 
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